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Rn RpproKimate Solution Technique for the Constrained Search Path 

Nouing Target Search Problem 

James N. Eagle 
James R. Vee 

Department of Operations Research 
Naoal Postgraduate School 
Honterey, Cfl 93943 

R discrete time search is conducted for a target mooing among a 
finite set of cells C = fit the beginning of each time 

period one cell is searched. If cell i mas searched in the preoious 
time period, the current search cell must be selected from the set 
Cj ^ C. If the target is in the selected cell k, it is detected 
( i . e. , found) m i th probabi lity q|<€[0,l3. If the target is not i n 
the cell searched, it can not be detected during the current time 
period. Rfter an unsuccessful search, a target in cell i mooes to 
cell j with probability T'jj for the next time period. The transition 
probability matrix F= [T'jj] and the initial distribution of the 
target ooer the search cells are knomn to the searcher. The 
object ioe of the searcher is to select a T-time period search path 
luhich minimizes the probability of nondetect ion. 

1. Background 

The path constrained search problem, described abooe, is a 
difficult one to soloe efficiently. Trummel and Ueisinger [1985] 
showed that the path constrained search problem with a stationary 
torget is NP-complete. The mooing target problem, which is a 
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generalization of the stationary target problem, is then also liP- 
complete. 

Other than total enumeration of all search paths, the only 
optimal solution technique mentioned in the literature for the 
mouing target constrained search problem has been the dynamic 
programming procedure of Eagle[1984a] . Although this method can 
solue problems much more quickly than total enumeration, it can 
require a large amount of computer storage as problem size 
increases. 

It mas the difficulty experienced uiith soloing large problems 
optimally that mot looted the deoelopment of good suboptima I solution 
procedures, The first such method proposed mas a modified branch- 
and-bound method by Stemart [1979], Stemart used a discrete oersion 
of a mooing target search algorithm gioen by Gromn [1960] to prooide 
bounds for his procedure, Homeoer, Bromn’s algorithm does not 
necessarily gioe optimal solutions mhen search effort is discrete, 
so these "bounds" may result in an optimal branch of the enumeration 
tree being mistakenly fathomed. Nonetheless, St smart's 
computational experience mith 1 -dimensional search problems 
indicates that the method can perform mell. 

Another approximate procedure mas giuen by Eagle [1981b]. This 
dynamic programming method uses a mooing or "rolling" time horizon 
that greatly reduces the computer storage requirements. It mas used 
to approximately solue a small 2-dimensional problem (3 by 3 search 
grid) for 10 time periods. This procedure generalizes myopic search 
by selecting in each time period the next cell to be searched under 
the assumption that the search ends m time periods in the future. 



For myopic search, m is 1. For small enough m, this procedure can 
be implemented on a microcomputer. 

Reported here is a third suboptima I solution method mhich, like 
Groum's algorithm, is derived from a nonlinear programming 
formulation of the search problem. Unlike the problem addressed by 
Broiun, houieuer, this formulation (a) alloujs for path constraints to 
be specified for the searcher, (b) does not alloiu search effort in 
each time period to be infinitely divisible over the search cells, 
and (c) does not have a convex detection function. Consequently 
the objective function of the nonlinear program is not necessarily 
convex and the solutions obtained may be local rather than global 
optima. But like Bromn's method, the structure of the problem 
allouis a simple implementation of the nonlinear programming solution 
technique. When considered uiithout path constraints, this procedure 
has similarities to discrete versions of both Broiun’s algorithm and 
those of Uashburn [1980] and [1983]. 



The movement of the searcher is described by a nonhomogeneous 
Harkov process. Let Sjj(t) be the probability that the searcher mill 

search cell j in the time period t, given that cell i uias searched in 
time period t-1. Then a seorch plan . S = {S(l ), . . . ,S(T)}, is a 
sequence of T HxH stochastic matrices satisfying 



2. Definitions 
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R deterininist ic search plan is a S composed entirely of ones 



and zeros. 

Pjj(t) is the joint probability that, after the search and 
target transition in time period t, the searcher is in cell i and 
the target is in cell j and has not been detected by the first t 
searches. 

p(0) € is the initial joint seorcher-taroet distribution 
and is assumed to be known by the searcher. 

Hote that Ey pjj(t) is the probability that the target has not been 

detected by the searches conducted in time periods 1 through t. 
Also, Pjj(t) con be calculated recursively from pjj(t-1)by 

conditioning on the searcher cell and the target cell, after the 
search and target transition in time period (t-1). Specifically, 



where (?jj is 1 if i = j and 0 if i*j. 

3. The Search Problem as a Nonlinear Program (NLP) 

Ue seek the solution of the following NLP 



Pjj(t) - (l-qj<5jj) ^ ^ > 



( 1 ) 






( 2 ) 



subject to 





s(t) >, a, t = 1, . . .,T 



(5) 
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where Pg(T) is calculated recursiuely from (1). The decision 

uariables are the NxN matrices S(1 ), . . . jS(T) . To solue this 
problem, p(0), (qi, ^ 'f'-^st be specified. 

The four propositions which follow establish certain properties 
of the aboue nonlinear program. It is noted that this problem, 
while hauing a relatiuely complicated object iue function, has 
constraints which are linear and highly structured. 

Proposition 1: The minimum of nonlinear program (2)-(5) is achieoed 

by a deterministic search plan. 

Proof: Let S* be an optimal search plan. (Such a plan exists 

since the objective function (2) is continuous in S and the 
set of feasible S defined by (3)-(5) is compact.) S* defines a 

T-time period, M-cell riarkov process defining probabi I ist iciy 
an optimal search path. Let • • • finite set of 

po s s ibie deterministic search paths generated by Each 
has an associated aj, € where the j^'^' component of a^ is the 
probability of nondetection given the searcher follows and 
the target starts in cell j. filso let 
^Ij “ 

be the probability that the target starts in cell j. Then the 
probability of nondetection given Ck i-‘ followed and an initial 
target distribution of T|€IR'^^^ is the dot product ria|,. How if 
P(^(j) is the probability of the searcher following path when 
S* is used, we have 
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Prob{nondetect ion 



i X]d*, 

where 

a* = argminaj^ (iiak)' 

So the deterministic search path associated with a* (and the 
deterministic search plan which generated it) is also optimal, 

Proposition 2: S is a deterministic search plan if and only if S is 

an extreme point of the linear constraints (3)-(5). 

Proof: If S is deterministic, then by definition all component 

matrices of S must be composed entirely of zeros and ones. 

Such an S can not be written as a strict convex combination of 
two other matrix series satisfying (3)-(5). So S is an 
extreme point, find if S is not deterministic, then some 
component matrix must contain a row vector with two or more 
components strictly between 0 and 1. This row vector can then 
be expressed as a strict convex combination of two other 
distinct and feasible row vectors, Thus S is not an extreme 
point . 

Proposition 3: The objective function (2) is linear in S when 

constrained to any edge of the simplex formed by (3)-(5), 

Proof; Lets' and S" be any two adjacent extreme point solutions 
of the constraints (3)-(5), Specifically, S' and S" are 
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identical stochastic (0,1) matrix series except in one rou) of 
one matrix. Let t be the time period where S’ and S" differ, 
and for any A€[0,1] let 

= AS’ + d-A)S" 

= S(1 ),..., S(t-1),AS'(t) + (1-A)S"(t),S(t + l ),..., S(T) 
Conditioning on the searcher's cell and the target’s cell 
before the search in time period t, we can write the objective 
function (2) evaluated at S^ as 

where <$ki '^i (1), and probability that the 

target is undetected by searches in time periods t+1,...,T 
given the searcher is in cell k and target is in cell L at the 

end of time period t. Writing 

S^k(t) = AS'ik(t) + (l-A)S“ik(t), 
and observiog that Pjj(t-I) and Qki(^‘^^) Q*''® ricit functions of A, 
shows (6) to be linear in A. 

Proposition 1: fill basic feasible solutions to the nonlinear 

program (2)- (5) are nondegenerate. 

Proof: The constraint matrix represented by (3) and (T) consists 

of HI linearly independent rows. So basic solutions will always 
have NT basic variables, lie argue that these basic variables 
will always be positive. Each constraint consists of nonnegative 
var i ab I es summ i ng to un i t y . So any f eas i b I e sol ut i on must 
involve at least one variable from each constraint. Since there 
are NT constraints and NT basic variables, any basic solution 



must involve a single basic variable in each constraint, find 
furthermore, since the constraints sum to unity, the value of 
each basic variable must be 1. 

4. applying the Conuen SimpieK Method (C$M) 

The CSn, uihich is a generalization of the simplex method, is 
applied to. nonlinear programs with linear constraints. If the 
current feasible solution is an extreme point of the constraints, 
then the CSI1 determines the rate of change of the objective function 
along the edges radiating from that extreme point. The current 
solution then moves along an edge luith the greatest initial rate of 
improvement. This movement continues until a local optimum is found 
along that edge or an adjacent extreme point is reached (whichever 
occurs first). If there is no edge along which the objective 
function can be improved, then the current extreme point is a Kuhn- 
Tucker point, and a local optimum has been found. The importance of 
Propoposit ion 1 is that it guarantees that an extreme point which 
does not satisfy the Kuhn-Tucker conditions can always be improved 
with a single iteration of the CSI1. 

The CSI1 is applicable to the nonlinear program (2)-(5) because 
the constraints are linear. Furthermore, its implementation is 
especially easy since from Proposition .3 the objective function is 
linear along edges of the simplex. Thus once movement along an edge 
is determined to reduce the objective function, this movement 
continues until an adjacent extreme point is reached. That is, no 
one-dimensional seorches for local minima along extreme edges are 
required. 
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The CSn can thus be implemented for this problem by starting at 
any feasible extreme point, eualuating the objective function at all 
adjacent extreme points, and moving directly to the adjacent extreme 
point with the smallest objective function value. 

The calculations required to evaluate the objective function 
(2) at any extreme point of the constraints are simplified uihen the 
seorcher's starting cell is knomn uiith certainty (the usual case for 
most search applications). This occurs because the searcher's 
starting cell, together with the deterministic search plan 
associated uiith an extreme point, uniquely determine a search oath 
s = (s(1 ), . . . ,s(T)), where s(t) is the searcher's cell in time 
period t. Rnd the probability of nondetection given s is 



where T] is the initial target distribution, 1 is a column vector of 
ones, and rs(t) is the target transition matrix F with row s(t) 
multiplied by (l-qs{t))' 

It is also^ observed that, although a general search plan S is 
an element of a deterministic search plan can be represented 

by a TxH integer matrix 3. Each element S(t,j) is the searcher's 
cell in time period t given the searcher's cell in time period t-l 
wos j. 5 represents a feasible search plan If and only if each 
5(t,j) is an element of Cj. 



T 




(?) 
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The CSrij specifically tailored for the search problem in (2)- 
(5), is the fol lowing: 



1. Specify s(l) and an initial feasible 5. 

2. Determine s = (s(1 ), . . . ,s(T)) from s(1) and 3. 

3. Calculate PNO(s) from (7) and set PtlD^in ~ PHD(s). 



1. For t=1 to T, 

a. Uary 5(t^s(t)) over all Cs(t) to generate 
candidate adjacent extreme points S'. 

b. For each 5‘, calculate the new search path s' 
and PND(s'). 

c. If PND(s') < PHDminj save 5' and s', and set 



PHDr„in= PND(s’) 



d. Cont inue to next t . 

5. If no improvement is achieved in PHOrni^ after t=T, then 
STOP. Otherwise define 3 as 3', s as s’, and return 



to step 1. 



Note that each main iteration of this algorithm (i.e.,step 1.) 
requires the calculation of PHD for 2j |Cj | search paths, where |Cj I 
is the number of cells in Cj. 
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5. The Starting Solution 



In this sect I on j a procedure for generating an initial feasible 
S is described, fl reasonable S would be one that corresponds to a 
myopic search path. R myopic search path is a path where the 
searcher moues from the cell searched in time period t to that 
accessible cell j with the largest qjT]j(t)^ where 

l-jj(t) = 2j Pij(t) 

= Prob{target in cell j and has not been 
detected by searches in time periods 1;...,t}. 

If two or more accessible cells have identical maximal oalues 
of ^1*'® inyopic policy^ as used here^ attempts to break 

the tie by selecting a cell with the minimum Euclidean distance (or 

some other reasonable norm) to a cell with the maximum value of 

qi'r|j(t). If a tie still exists, it is broken randomly. When the 

target is distant, this tie breaking procedure attempts to generally 
move the searcher towards a favorable cell and, hopefully, into 
position for a future detection. 

fi procedure is still needed to complete the initial feasible S. One 
procedure would be to assume that the target distribution is a stationary 
distribution generated by the target transition matrix T. (That is, the 

target distribution i s where T) = TjF, r|j^0.) Then for each time 

period (i.e., each row of S) the myopic policy is used to determine the 
next cell to be searched. This results in an S which, except for one- 
element in each row, has identical rows. The element which is different 



corresponds to the initial myopic search path. This starting solution mas 
tested and, for the problems examined, appeared to mork mell. 



6. Three EHamples 
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Figure 1 . 25 cell search grid. 



The target and searcher moue among the 25 cells of Figure. 1. 

In the next time period, the searcher has access to the cell just 
searched plus all adjacent cells. Cells are adjacent if they share 
a common side. Thus, for example, = {1,2,6} and 
Cj = (2, 3, 1,6}. 

In each target transition, the target remains in the preuiously 
occupied cell j mith probability .1 and mooes to an adjacent cell 
with probability .6/mj, where mj is the number of cells adjacent to 

cell j . 

In the first two examples, the searcher starts in cell 1 and 
the target in cell 13. Resume that detection is certain if the 
target's cell is searched. That is, q|^ = 1, k = 1,2,..., 25. 
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R 10-tlme period problem mas soloed using FORTRRH 77 on an IBfl 
3033 mainframe computer. The fol lowing results mere obtained: 



Prob. of CPU time 

Search path Hondetection (sec) 

nyopic 6 7 12 13 H 13 18 19 H 9 .4977 .1 

CSM 67 8 13 14 19 18 17 12 7 .4886 1.42 



In addition^ a total enumeration routine mas written to find 

the optimal solutions. This procedure required 25 CPU minutes to 

examine the 1,225^623 possible 10-time period search paths^ and 

showed the CSI1 solution to be one of 12 distinct optimal solutions. 

In this example the myopic policy was close to optimal. Using 

the CStI resulted in only a slight improvement, while requiring 

considerably more CPU time. The next examples show these 

observations are not true in general. 

In the second example, a "fast" target was considered. 

Specifically, the probability that the target in cell j remained in 

j was 0, and the probability that a transition occured to any of the 

mj adjacent cells was 1/mj. fill other problem parameters remained 

the same. This problem gaive the following results: 

Prob. of CPU time 

Search path Nondetection (sec) 

llyopic 6 7 12 12 7 8 9 14 19 18 .4540 .1 

CSM 67 7 8 91419181712 . 3868 1 . 73 

Rgain, total enumeration showed the CSM solution to be optimal. 

The third example examined the possible consequences of a 
moving mean target position. Here the target started in cell 21 and 
moved up or right one cell, each with probability .45. The target 



remained in its current cell mith probability .1. This mouement up 
and right continued until either the top or right boundary mas 
reached. Then mith probability .9, the target mooed one cell up (if 
currently in cell 25 j 20 15j or 10) or one cell to the right (if in 
cell 1, 2, 3, or 1). Again, the target remained in the current cell 
with probability .1. Uhen the target reached cell 5, it remained 
there foreoer and was assumed to haoe escaped. Except in cell 5, 



detection was certain if the target's cell was searched. In cell 5 
the target could not be detected. For the 10-time period problem, 



the following 


results were 


obt 


ained: 
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.2517 


.1 


CSM 


6 11 
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15 
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10 


10 


.1531 


1 .09 


Total 


enumerat i on again 


showed 


the CSn 


path to he opt imal . 


In this 



final example, the stationary target distribution has the target in 
cell 5 (the trapping state) with certainty. Since qg is 0, the 

modified myopic procedure given in the previous section fails to 
give a unique starting S. To find a starting solution for this 
example, it was assumed that qg was an orbitrarily smoll, positive- 
number . 

Finally it is noted that, like most nonlinear programming 
solution procedures, using a poor starting solution can result in a 
poor final solution. In the last example, setting S-,j = j (except 

for the one element of each row determined by the myopic path) 
resulted in a local optimal PHD of .2379, a slight improvement over 
the myopic solut ion. 
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